IndexOutOfBoundsException while call suggest api #3196

piaolingxue · 2013-06-17T08:41:09Z

java.lang.IndexOutOfBoundsException: Index: 2, Size: 2
at java.util.ArrayList.rangeCheck(ArrayList.java:571)
at java.util.ArrayList.get(ArrayList.java:349)
at org.elasticsearch.search.suggest.Suggest$Suggestion.reduce(Suggest.java:251)
at org.elasticsearch.search.suggest.Suggest.reduce(Suggest.java:179)
at org.elasticsearch.action.suggest.TransportSuggestAction.newResponse(TransportSuggestAction.java:145)
at org.elasticsearch.action.suggest.TransportSuggestAction.newResponse(TransportSuggestAction.java:60)
at org.elasticsearch.action.support.broadcast.TransportBroadcastOperationAction$AsyncBroadcastAction.finishHim(TransportBroadcastOperationAction.java:369)
at org.elasticsearch.action.support.broadcast.TransportBroadcastOperationAction$AsyncBroadcastAction.onOperation(TransportBroadcastOperationAction.java:306)
at org.elasticsearch.action.support.broadcast.TransportBroadcastOperationAction$AsyncBroadcastAction.performOperation(TransportBroadcastOperationAction.java:265)
at org.elasticsearch.action.support.broadcast.TransportBroadcastOperationAction$AsyncBroadcastAction.performOperation(TransportBroadcastOperationAction.java:242)
at org.elasticsearch.action.support.broadcast.TransportBroadcastOperationAction$AsyncBroadcastAction$1.run(TransportBroadcastOperationAction.java:218)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

s1monw · 2013-06-17T08:51:26Z

can you provide an example request?

piaolingxue · 2013-06-20T09:24:13Z

On 06/17/2013 04:51 PM, Simon Willnauer wrote:

can you provide an example request?

—
Reply to this email directly or view it on GitHub
#3196 (comment).

for example
run this script will throw Exception
like:java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
but if I compile elasticsearch locally, then run the es server and this
script, it will not throw exception.
/#!/bin/bash//
//
//echo "delete index[products]"//
//curl -X DELETE 0:9200/products//
//echo "create index[products]"//
//curl -X PUT 0:9200/products -d '//
//{//
// "settings" : {//
// "analysis" : {//
// "tokenizer" : {//
// "ngram_min_1" : {//
// "type" : "nGram",//
// "min_gram" : "1",//
// "max_gram" : "10"//
// }//
// },//
// "analyzer" : {//
// "ngram_min_1" : {//
// "type" : "custom",//
// "tokenizer" : "ngram_min_1"//
// }//
// }//
// }//
// },//
// "mappings" : {//
// "product" : {//
// "properties" : {//
// "ProductId": { "type": "string", "index":
"not_analyzed" },//
// "ProductName" : {//
// "type": "string", //
// "index_analyzer": "whitespace",//
// "search_analyzer" : "ngram_min_1"//
// }//
// }//
// }//
// }//
//}'//
//echo//
//echo "add docs"//
//for i in 1 2 3 4 5 6 7 8 9 10 100 101 1000; do//
// json=$(printf '{"ProductId": "%s", "ProductName": "%s" }', $i, "My
Product $i")//
// echo $json//
// curl -X PUT 0:9200/products/product/$i -d "$json"//
// echo//
//done//
//
//echo//
//echo 'seggestion with:prodct'//
//curl -XPOST '0:9200/_suggest' -d '{//
// "text" : "prodct",//
// "my-seggestion1" : {//
// "term" : {//
// "field" : "ProductName",//
// "suggest_mode" : "always"//
// }//
// }//
//}'/

martijnvg · 2013-06-20T12:04:41Z

So you build ES from source based on master or based on the 0.90 branch? What Elasticsearch version did initially use when you ran the shell script?

piaolingxue · 2013-06-21T01:54:43Z

On 06/20/2013 08:05 PM, Martijn van Groningen wrote:

So you build ES from source based on master or based on the 0.90
branch? What Elasticsearch version did initially use when you ran the
shell script?

—
Reply to this email directly or view it on GitHub
#3196 (comment).

I build ES based on the 0.90.1 branch (refs/tags/v0.90.1)
and I firstly ran the shell script use the 0.90.1 version.

My OS is Ubuntu 12.04，have you met this problem?

piaolingxue · 2013-06-21T01:56:34Z

On 06/20/2013 08:05 PM, Martijn van Groningen wrote:

So you build ES from source based on master or based on the 0.90
branch? What Elasticsearch version did initially use when you ran the
shell script?

—
Reply to this email directly or view it on GitHub
#3196 (comment).

and my JVM info as follows:
java version "1.6.0_27"
OpenJDK Runtime Environment (IcedTea6 1.12.3) (6b27-1.12.3-0ubuntu1~12.04.1)
OpenJDK 64-Bit Server VM (build 20.0-b12, mixed mode)

martijnvg · 2013-06-24T15:48:12Z

@piaolingxue I tried to run your script, but I don't run into the IndexOutOfBoundsException error.

gromgull · 2013-07-24T10:01:37Z

I've also had this exception (v. 0.90.2), it is unfortunately not consistently reproducable, the same query will sometimes work ok, and sometimes break.

I can offer two hints though:

I think it only happens when there are at least two terms in the suggest text
I would guess it's related to using a different field/analyzer for the query and for the suggestions? My content field is a multi_field, once with a snowball stemmer (this one is queried), and once with a standard analyzer for the suggestions.

I will keep trying to make a contained test that breaks at least sometimes :)

s1monw · 2013-07-24T16:24:13Z

@gromgull is it the same stacktrace as above?

gromgull · 2013-07-25T07:36:17Z

Close, the underlying exception is nearly identical:

2013-07-24 11:30:03,532][DEBUG][action.search.type       ] [Ghazikhanian, Carter] failed to reduce search
org.elasticsearch.action.search.ReduceSearchPhaseException: Failed to execute phase [fetch], [reduce]
        at org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction.finishHim(TransportSearchQueryThenFetchAction.java:177)
        at org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction$3.onResult(TransportSearchQueryThenFetchAction.java:155)
        at org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction$3.onResult(TransportSearchQueryThenFetchAction.java:149)
        at org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteFetch(SearchServiceTransportAction.java:346)
        at org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction.executeFetch(TransportSearchQueryThenFetchAction.java:149)
        at org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction$2.run(TransportSearchQueryThenFetchAction.java:136)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
        at java.lang.Thread.run(Thread.java:680)
Caused by: java.lang.IndexOutOfBoundsException: Index: 3, Size: 3
        at java.util.ArrayList.RangeCheck(ArrayList.java:547)
        at java.util.ArrayList.get(ArrayList.java:322)
        at org.elasticsearch.search.suggest.Suggest$Suggestion.reduce(Suggest.java:246)
        at org.elasticsearch.search.suggest.Suggest.reduce(Suggest.java:174)
        at org.elasticsearch.search.controller.SearchPhaseController.merge(SearchPhaseController.java:404)
        at org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction.innerFinishHim(TransportSearchQueryThenFetchAction.java:190)
        at org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction.finishHim(TransportSearchQueryThenFetchAction.java:175)
        ... 8 more

s1monw · 2013-12-15T19:49:45Z

Eventually I was able to reproduce this! The reason here seems to be that there is more than one index and the indices share a field with the same name but different analysis chains that produce different number of tokens. It would be great if anybody could confirm that their setup also has more than one index were at least 2 indices share the same field with different analysis chains. I added a commit to a branch with a test that reproduces this issue.

s1monw · 2013-12-15T20:05:42Z

Added a fix for this in the commit above. IMO we should throw an exception and fail the reduce if the size is not the same. @martijnvg @imotov what do you think?

imotov · 2013-12-15T21:45:18Z

Looks good to me.

martijnvg · 2013-12-16T09:16:30Z

LGTM as well.

s1monw · 2013-12-16T13:06:57Z

@martijnvg @imotov I force pushed another change that also checks the entry keys since we also need to make sure we can merge them. There is also a test that checks it. Can you take another look?

If the term suggester is used the results are merged depending on the number of terms produced by the tokenizer / tokenfilter. If a term suggester is executed across multiple indices that share the same field but with different analysis chains we can't merge the result anymore sicne tokens are our of order or have a different size. This commit throws ESIllegalArgumentException if the number of entries are not the same across all results. Closes elastic#3196

If the term suggester is used the results are merged depending on the number of terms produced by the tokenizer / tokenfilter. If a term suggester is executed across multiple indices that share the same field but with different analysis chains we can't merge the result anymore sicne tokens are our of order or have a different size. This commit throws ESIllegalArgumentException if the number of entries are not the same across all results. Closes #3196

If the term suggester is used the results are merged depending on the number of terms produced by the tokenizer / tokenfilter. If a term suggester is executed across multiple indices that share the same field but with different analysis chains we can't merge the result anymore sicne tokens are our of order or have a different size. This commit throws ESIllegalArgumentException if the number of entries are not the same across all results. Closes elastic#3196

ghost assigned s1monw Dec 15, 2013

s1monw closed this as completed in 3e32197 Dec 16, 2013

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

IndexOutOfBoundsException while call suggest api #3196

IndexOutOfBoundsException while call suggest api #3196

piaolingxue commented Jun 17, 2013

s1monw commented Jun 17, 2013

piaolingxue commented Jun 20, 2013

martijnvg commented Jun 20, 2013

piaolingxue commented Jun 21, 2013

piaolingxue commented Jun 21, 2013

martijnvg commented Jun 24, 2013

gromgull commented Jul 24, 2013

s1monw commented Jul 24, 2013

gromgull commented Jul 25, 2013

s1monw commented Dec 15, 2013

s1monw commented Dec 15, 2013

imotov commented Dec 15, 2013

martijnvg commented Dec 16, 2013

s1monw commented Dec 16, 2013

IndexOutOfBoundsException while call suggest api #3196

IndexOutOfBoundsException while call suggest api #3196

Comments

piaolingxue commented Jun 17, 2013

s1monw commented Jun 17, 2013

piaolingxue commented Jun 20, 2013

martijnvg commented Jun 20, 2013

piaolingxue commented Jun 21, 2013

piaolingxue commented Jun 21, 2013

martijnvg commented Jun 24, 2013

gromgull commented Jul 24, 2013

s1monw commented Jul 24, 2013

gromgull commented Jul 25, 2013

s1monw commented Dec 15, 2013

s1monw commented Dec 15, 2013

imotov commented Dec 15, 2013

martijnvg commented Dec 16, 2013

s1monw commented Dec 16, 2013